This invention relates in general to video systems and, more specifically, to aspect ratio manipulation of video images.
From its standardization in the 1950's, television displays have traditionally used a 4:3 aspect ratio, displaying an image that is four units wide and three units high. A very large quantity of television programs were produced in the 4:3 aspect ratio. In 1996, the FCC adopted new display formats for digital television and chose a 16:9 aspect ratio display for high definition television (HDTV). Beginning in 2002, some television production began shifting from the traditional 4:3 aspect ratio images to 16:9 images in an attempt to “future proof” the product because a 16:9 image may be processed for display on a legacy 4:3 display by cutting off the sides in a process called “pan and scan.” Using pan and scan, the center part of the wider aspect ratio image is cropped to produce a 4:3 aspect ratio image. Another conventional technique is the “bob and crop” and is used to convert aspect ratios.
In FIGS. 1 and 2, a conventional 16:9 display 100 and a 16:9 image 200 are respectively shown. In the image depictions of the figures, hatched circle images are used accentuate any distortion to the image. FIGS. 3 and 4 respectively show a conventional 4:3 display 300 and a conventional 4:3 image 400. Today, conventional televisions are available in either format, but other aspect ratios are known. For example, movie theatres may display even wider images.
The pan and scan process uses trained technicians to manually crop a center part of the image with dynamic adjustments of a 4:3 aperture left or right to capture most of the content of the 16:9 image 200. In other words, the edges of the 16:9 image 200 are cut off to fill the 4:3 display 300. In a similar process, a 16:9 aperture pans and scans up and down through a 4:3 image 400, but cuts off horizontal slices of the 4:3 image 400 below and above the 16:9 aperture.
During the transition to digital television there will be a time when broadcasters will transmit signals with both 4:3 and 16:9 images 400, 200. Various consumers will have both 4:3 and 16:9 television sets 300, 100 that respectively display both 4:3 and 16:9 images 400, 200. The optimum viewer experience will exist when the display aspect ratio exactly matches the image aspect ratio, for example a 4:3 display 300 showing a 4:3 image 300 or a 16:9 display 100 showing a 16:9 image 200. In both these cases, the image fills the display and the image is presented with all the available horizontal and vertical resolution possible in the display.
Often the image aspect ratio of the image doesn't match the aspect ratio of their TV display. This will be an ongoing problem because of the large amount of television programs produced in 4:3 aspect ratio. On the other hand, theatrical movies are produced in widescreen format with an aspect ratio of 1.85 or wider, which causes aspect ratio problems also.
There are two conventional ways to present a 16:9 image 200 on a 4:3 display 300, as shown in FIGS. 5 and 6. FIG. 5 uses letterbox format 500 to match the width of the 4:3 display with the width of the 16:9 image 200, producing black bands 508 at the top and bottom of the 4:3 display 300. The 16:9 image height is reduced to 75% of the 4:3 screen height, and the overall displayed image 200 is reduced to 75% of the display area. Because of the fixed vertical scanning structure (physical pixels on LCD and DLP sets, or scan lines on CRT-based displays), available vertical resolution is effectively reduced 25% by the letterbox display 500.
Another problem with the letterbox display 500 is consumer dissatisfaction. When the image doesn't fill the screen, most consumers feel like they are being short-changed, that they are not getting the full value from their television sets. And the black bands 508 are distracting from the viewing experience. Yet another problem with the letterbox display 500 is “burn in” on some types of displays. Since some display elements (or phosphors) are not used in the letterbox display 500, the display elements age differently, and can result in the black bands 508 being “burned in” to the display 500, showing a persistent band even when not using letterbox format.
The second conventional approach to fill the 4:3 display with a 16:9 image is to stretch the image vertically as shown in FIG. 6. This approach would use the entire vertical scanning capability of the 4:3 display 300 to deliver the maximum available vertical resolution. In practice, this approach produces unacceptable geometric distortion which vertically elongates the image 604.
In conventional systems, 4:3 images 400 are commonly shown on 16:9 displays 100 using a pillar box presentation 700, with black vertical bars 708 on the left and right of the image 400. Using this approach that is shown in FIG. 7, the 4:3 image 400 only uses 75% of the available 16:9 screen width. In other words, the 4:3 image 400 uses only 75% of the 16:9 display area. The horizontal resolution is reduced by 25% since every scan line of the image 400 does not utilize all the available pixels of a scan line of the 16:9 display 100. The previous problems with consumer satisfaction and burn-in discussed above also apply to the pillar box display 700. In this case, the vertical pillar boxes 708 will be burned in.
Another approach 800, shown in FIG. 8, is to stretch the 4:3 image 804 to match the width of the 16:9 display 100. The resulting geometric distortion may be tolerable for some kinds of viewing, but the geometric distortion is always present nonetheless.
In conventional systems, some variations on the horizontal stretch apply a more pronounced stretch at the left and right edges of the image, and apply less stretch in the center of the picture. This non-linear type of stretch leaves the geometry in the center of the image relatively unchanged but at the expense of more radical geometric distortions at the left and right edges.
In the appended figures, similar components and/or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.